Your home is where your loved ones sleep. Your home is where your most cherished belongings are stored. Your home must be safe. The objective of this project is to build a Smart Home Security System. This enables deterrence through accountability, protecting your home and everything it stands for. Objectives include:
This project is to build a Smart Home Security System that is cheap,
robust and easy to install. It can be wall-powered or battery powered,
with access to an Internet connection (Ethernet or WiFi) as the only
requirement. The goal of this project is to provide safety and peace of
mind to the owner at an affordable cost. The system requires minimal
maintenance and is very easy to operate.
All peripherals and sensors are connected to the Raspberry Pi 3 Model B
device. The Raspberry Pi Camera Module is mounted within a pan-tilt
hardware kit with two degrees of rotation enabling 180º of side-to-side
(pan) rotation and 150º of up-and-down (tilt) rotation. The camera
movement is user controllable through an online web interface, which
greatly expands the effective field of view.
A Passive Infrared (PIR) sensor is coupled with a computer vision
algorithm to detect motion. When a threat is detected, an email is
immediately sent to alert the owner. The email notification contains a
link to the web interface and an image identifying the threat.
This system can be broken down into five parts:
The difficulty of this project was compounded due to COVID-19 pandemic. Tyler had all of the hardware, but unlike on campus, he only had access to a very limited toolset (i.e., wire stripper) and did not have any circuit equipment (multimeter, oscilloscope, etc.). Nonetheless, we were able to connect all of our hardware as detailed in the circuit diagram below. Clearly, we did not use the piTFT display. This was a conscious decision because of the embedded nature of our device. Instead of the piTFT display, a website was designed as the user information and control mechanism.
R-Pi Camera Module V2
Example of pan-tilt rotation
HC-SR501 PIR Sensor
The servos are connected to the hardware PWM GPIO pins (18 and 19). A switch is inserted between the servos and the 6V battery pack (four AA batteries) to enable easily disconnecting the servos when not in use, preserving the batteries.
Current limiting resistors (1kΩ) are used between the external devices and the GPIO pins. This is essential because we are controlling hardware with software. The current limiting resistors are used to protect the GPIO pins from problematic software mistakes (accidentally programming the input pin as an output). The current limiting resistors would keep the generated current within the tolerance of the GPIO pins even if such a mistake were to occur.
A simple solution—double sided adhesive foam tape—was used to mount the R-Pi Camera Module V2 to the pan-tilt base. While we initially planned to make a nice laser cut container to package all of the hardware, the pandemic made this impossible.
Circuit schematic
The first thing we did is develop a comprehensive set of tools to remotely manage the Raspberry Pi system. The R-Pi system is located at Tyler’s house with an HDMI monitor, mouse and keyboard. On Tyler’s local network, the R-Pi has the static IP address 192.168.7.194. We used Dynamic DNS by NoIP to enable accessing Tyler’s home network using a hostname. Combined the port forwarding on Tyler’s home router, this enabled global access to the R-Pi. Additionally, we turned on RealVNC to enable access to the R-Pi desktop.
To achieve a high frame rate and low latency video stream, we used "Advanced Recipe 4.10 - Web Streaming" from the PiCamera documentation. This is a simple HTTP server that achieves significantly higher frame rates than anything else we tested (mjpg-streamer, Motion program, RPi-Cam-Web-Interface). This is implemented in webStream.py. It is the only process to directly interact with the camera. The video stream is served to the other programs that require it. The video is streamed at 60 fps with a resolution of 640x480 pixels.
To implement our computer vision algorithm for motion detection, we used the Motion program. Motion is a highly configurable program that monitors video signals from many camera types. In our case, we pass through the video stream from webStream.py. The motion detection algorithm looks at the number of changed pixels in consecutive frames. A motion event is triggered when greater than 5% of the total pixels in the frame are different, after filtering and noise reduction. To trigger a motion detection alert, three consecutive frames must contain a motion event. A motion detection alert is ended after 10 consecutive seconds of no motion detected.
The pir.py python program performs motion detection using the PIR sensor. This function uses callbacks due to their improved efficiency over polling. Importantly, this function includes a 30 second warm up period in which the PIR sensor acclimates to the environment. This initialization period is essential for proper operation.
When motion is detected, whether by PIR sensor or the computer vision algorithm, an email alert is sent to the user. The email contains a hyperlink to take the user directly to the livestream. If the motion alert was triggered by the computer vision algorithm, it will also include an image from the video stream with a box drawn to identify the motion. This is achieved using the Mutt utility, which allows you to send an HTML file as the body of the email. msmtp, an SMTP client, and a free Google email address are used.
We followed an incremental testing approach that mirrored our development plan. This enabled us to parallelize our work, ensuring each individual component was fully functional before being integrated into the system-at-large. During week 2, we spent a lot of time familiarizing ourselves with the configuration options for the Motion program. This let us iterate our motion detection thresholds without relying on other programs. During week 2 we also figured out how to send automated emails, and we practiced this in various configurations. Doing this early on made it trivial to integrate later.
Testing the pan-tilt hardware was complicated by the lack of tools (multimeter, wires, etc.) available, but we took it slow and made it work. Using the PiGPIO library and Pygame, we built a simple remote to control the servos. This greatly aided in our debugging of the system.
In addition, we created programs to test launching background processes, building a Flask web server and streaming camera footage. All testing files are located in the GitHub repository under /Project Code/Testing. In the end, we “stress tested” our system by having it run for an extended period of time. During this, all the various user control functions were tested to ensure proper operation.
We faced numerous issues throughout our project. This included:
Our complete hardware list is included in the below Bill of Materials (BOM). Our total budget is well below the $100 per team limit.
Component | Vendor | Part Number | Cost |
---|---|---|---|
Raspberry Pi 3 Model B Rev 1.2 | Adafruit | 3055 | - |
piCobbler Expansion Cable | Adafruit | 914 | - |
16GB Micro SD Card | SanDisk | - | - |
Raspberry Pi Camera Board v2 - 8 Megapixels | Adafruit | 3099 | - |
Passive Infrared (PIR) Motion Sensor | DIYmall | HC-SR501 | - |
Pan/Tilt/Roll Camera Mount | Fat Shark | FSV1603 | $59.99 |
Total Cost = | $59.99 |
After countless hours, many tests, lots of research and despite the
circumstances, all aspects of the Smart Home Security System work as
expected. The system gracefully starts using the launch script.
Likewise, clicking the shutdown button on the website gracefully tears
the system down and kills all the background processes. The video
stream has been optimized to minimize latency and the motion detection
thresholds have been refined through numerous trials. Emails are
automatically sent to alert the user when motion is detected.
The result of our work is a responsive, globally accessible website
interface that displays the video stream and the motion detection
status of the PIR sensor. Additionally, the website allows the user to
control the camera movement and shutdown the system. For security
purposes, the website is protected using a username and password.
The user control website interface looks as follows:
When the PIR sensor detects motion, the Motion Detection section of the website is asynchronously updated as follows:
There are many avenues for further exploration and improvement. Areas include:
The beginning of this project coincided with the emergence of the COVID-19 pandemic in the U.S. The circumstances enabled us to learn a great deal about remote management systems and networking. All circuitry work was done with limited tools and zero equipment, illustrating what is possible when you improvise and have an open mind and flexible attitude. Despite the incredible circumstances, we achieved all the objectives outlined in our project proposal. We implemented a fully functional embedded home security system on a resource constrained hardware platform. The system is globally accessible, has a responsive web interface, performs motion detection using both a PIR sensor and a computer vision algorithm and provides instant motion detection alerts via an automated email mechanism. A significant portion of this project required researching, understanding, installing and modifying various third-party APIs to support our hardware, which was a great learning experience for our future careers. Amazingly, this powerful system is extremely cheap!
We would like to thank Professor Joe Skovira and TAs Alex Hatzis, Caitlin Stanton, Canhui Yu and Sophie He for their help and guidance throughout the semester. Without them, this project would not have been nearly as successful. Additionally, we’d like to acknowledge the Linux and Raspberry Pi community at-large for providing a platform for low cost hardware projects and enabling hobbyists and tinkerers to pursue their passion.
All of our project code, including the code for this website, is hosted publically on GitHub at: ECE 5725 Final Project. Code for this website is in the docs folder and the project source code is in the Project Code folder.
Due to the large number of software files, only core code is included here. The complete code directory is available in the GitHub repository. Account usernames and passwords have been redacted where applicable.
launch.sh
Script to elegantly launch the system
#!/bin/bash
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/22/2020 #
# Lab 5: Final Project #
#################################################
# Use 'chmod +x launch.sh' to make executable
# Start PiGPIO daemon
# Daemon must be running for library to work
if pgrep pigpiod
then
echo 'PiGPIO daemon running'
else
sudo pigpiod
fi
# Launch rapid camera network stream
# Use absolute path
python3 ~/FinalProject/webStream.py &
# Launch PIR sensor
# Use absolute path
python3 ~/FinalProject/pir.py &
# Launch servo controller
# Use absolute path
python3 ~/FinalProject/FlaskWebServer/static/scripts/servoControl.py &
# Launch Motion program
# -c : full path & filename of config file
# -b : run in daemon mode
motion -b -c ~/FinalProject/Motion/motion.conf &
# Launch Flask WebServer
# Use absolute path
python3 ~/FinalProject/FlaskWebServer/website.py
#---------TERMINATION & CLEANUP---------#
# Kill the Motion program
if pgrep motion
then
sudo service motion stop
echo 'Motion program killed'
else
echo 'Motion program not running'
fi
# Kill the python processes
if pgrep python
then
killall python3
killall python
echo 'Python processes killed'
else
echo 'Python not running'
fi
# Kill the PiGPIO daemon
if pgrep pigpiod
then
sudo killall pigpiod
echo 'PiGPIO daemon killed'
else
echo 'PiGPIO daemon not running'
fi
website.py
Flask web server
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/22/2020 #
# Lab 5: Final Project #
#################################################
from flask import Flask, render_template, request, url_for, copy_current_request_context
from flask_basicauth import BasicAuth
from flask_socketio import SocketIO, emit
from time import sleep
from threading import Thread, Event
import os
app = Flask(__name__)
# Forces username:password login for all pages
app.config['SECRET_KEY'] = 'XXXXXXXXXXXXXX' #redacted
app.config['DEBUG'] = False
app.config['BASIC_AUTH_USERNAME'] = 'XXXXX' #redacted
app.config['BASIC_AUTH_PASSWORD'] = 'XXXXX' #redacted
app.config['BASIC_AUTH_FORCE'] = True
basic_auth = BasicAuth(app)
# Turn Flask app into a SocketIO app
socketio = SocketIO(app, async_mode=None, logger=True, engineio_logger=True)
# Create thread for PIR sensor
thread = Thread()
thread_stop_event = Event()
# Function that asynchronously updates PIR status using named pipe
def pirSensor():
# initial setup
prevPirStatus = 0
setupCount = 0
while not thread_stop_event.isSet():
f = open("/home/pi/FinalProject/FlaskWebServer/static/scripts/pir.txt", "r")
pirStatus = int(f.read()[0:1])
f.close()
# PIR detected motion --> changed to high
if(prevPirStatus==0 and pirStatus==1):
socketio.emit('pirStatus', {'pir': 'Detected'}, namespace='/pir')
prevPirStatus = 1
# PIR has no detected motion --> changed to low
elif(prevPirStatus==1 and pirStatus==0 or pirStatus==0 and setupCount<=50):
socketio.emit('pirStatus', {'pir': 'Clear'}, namespace='/pir')
prevPirStatus = 0
setupCount += 1
socketio.sleep(1)
@app.route("/", methods=['GET', 'POST'])
def homepage():
templateData = {
}
# Respond to button presses
if request.method == 'POST':
if request.form['form'] == 'Shutdown':
os.system('exec ~/FinalProject/FlaskWebServer/static/scripts/shutdown.sh')
if request.form['form'] == 'left':
os.system('python ~/FinalProject/FlaskWebServer/static/scripts/servoLeft.py')
if request.form['form'] == 'right':
os.system('python ~/FinalProject/FlaskWebServer/static/scripts/servoRight.py')
if request.form['form'] == 'up':
os.system('python ~/FinalProject/FlaskWebServer/static/scripts/servoUp.py')
if request.form['form'] == 'down':
os.system('python ~/FinalProject/FlaskWebServer/static/scripts/servoDown.py')
if request.form['form'] == 'center':
os.system('python ~/FinalProject/FlaskWebServer/static/scripts/servoCenter.py')
return render_template('index.html', **templateData, scrollToAnchor='servo')
return render_template('index.html', **templateData)
@socketio.on('connect', namespace='/pir')
def test_connect():
# Need visibility of the global thread object
global thread
print('Client connected')
#Start PIR sensor thread only if thread has not been started
if not thread.isAlive():
print("Starting Thread")
thread = socketio.start_background_task(pirSensor)
@socketio.on('disconnect', namespace='/pir')
def test_disconnect():
print('Client disconnected')
if __name__ == '__main__':
socketio.run(app, host='0.0.0.0', port=8001, debug=False)
webStream.py
Create video feed from PiCamera
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/22/2020 #
# Lab 5: Final Project #
#################################################
'''
- https://picamera.readthedocs.io/en/release-1.13/recipes2.html#web-streaming
- Run using python3
- 640x480 @ 60 fps streamed to port 8000
'''
import io
import picamera
import logging
import socketserver
from threading import Condition
from http import server
PAGE="""\
PiCamera MJPEG Video Stream
"""
class StreamingOutput(object):
def __init__(self):
self.frame = None
self.buffer = io.BytesIO()
self.condition = Condition()
def write(self, buf):
if buf.startswith(b'\xff\xd8'):
# New frame, copy the existing buffer's content and notify all
# clients it's available
self.buffer.truncate()
with self.condition:
self.frame = self.buffer.getvalue()
self.condition.notify_all()
self.buffer.seek(0)
return self.buffer.write(buf)
class StreamingHandler(server.BaseHTTPRequestHandler):
def do_GET(self):
if self.path == '/':
self.send_response(301)
self.send_header('Location', '/index.html')
self.end_headers()
elif self.path == '/index.html':
content = PAGE.encode('utf-8')
self.send_response(200)
self.send_header('Content-Type', 'text/html')
self.send_header('Content-Length', len(content))
self.end_headers()
self.wfile.write(content)
elif self.path == '/stream.mjpg':
self.send_response(200)
self.send_header('Age', 0)
self.send_header('Cache-Control', 'no-cache, private')
self.send_header('Pragma', 'no-cache')
self.send_header('Content-Type', 'multipart/x-mixed-replace; boundary=FRAME')
self.end_headers()
try:
while True:
with output.condition:
output.condition.wait()
frame = output.frame
self.wfile.write(b'--FRAME\r\n')
self.send_header('Content-Type', 'image/jpeg')
self.send_header('Content-Length', len(frame))
self.end_headers()
self.wfile.write(frame)
self.wfile.write(b'\r\n')
except Exception as e:
logging.warning(
'Removed streaming client %s: %s',
self.client_address, str(e))
else:
self.send_error(404)
self.end_headers()
class StreamingServer(socketserver.ThreadingMixIn, server.HTTPServer):
allow_reuse_address = True
daemon_threads = True
with picamera.PiCamera(resolution='640x480', framerate=60) as camera:
output = StreamingOutput()
camera.rotation = 180
camera.start_recording(output, format='mjpeg', quality=5)
try:
address = ('', 8000)
server = StreamingServer(address, StreamingHandler)
server.serve_forever()
finally:
camera.stop_recording()
msmtprc
Email configuration file
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/09/2020 #
# Lab 5: Final Project #
#################################################
# User configuration file ~/.msmtprc
# msmtp is an SMTP client: https://marlam.de/msmtp/
# REFERENCES
# https://marvintan.com/posts/send-email-using-google-stmp/
# Use MSMTP because SSMTP doesn't work on Buster
# https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=244147
# Set default values for all following accounts
defaults
# Use the mail submission port 587 instead of the SMTP port 25
port 587
# Always use TLS
tls on
tls_starttls on
tls_trust_file /etc/ssl/certs/ca-certificates.crt
# User specific log location, otherwise use /var/log/msmtp.log, however,
# this will create an access violation if you are user pi, and have not changes the access rights
logfile ~/.msmtp.log
# ----------------------------------------------- #
# Gmail service specifics #
# ----------------------------------------------- #
account gmail
# Host name of the SMTP server
host smtp.gmail.com
# From address
from ECE 5725 Security Camera
# Authentication: the password is given below
auth on
user XXXXXXXXXXXXXXXXXXXXXXX #redacted
password XXXXXXXXXXXXXXXXXXXXXXX #redacted
# Set a default account
account default : gmail
pir.py
Monitor the PIR sensor using callbacks
#!/usr/bin/env python
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/20/2020 #
# Lab 5: Final Project #
#################################################
import RPi.GPIO as GPIO
import time
import subprocess
# Use Broadcom Numbering system
GPIO.setmode(GPIO.BCM)
PIR_PIN = 23
GPIO.setup(PIR_PIN, GPIO.IN)
# Define a threaded callback function to run in another thread when events are detected
def MOTION(PIR_PIN):
if GPIO.input(PIR_PIN): # GPIO23 == high
msg = 'echo 1 > /home/pi/FinalProject/FlaskWebServer/static/scripts/pir.txt'
subprocess.check_output(msg, shell=True)
email = 'mutt -e "set content_type="text/html"" tss86@cornell.edu -s "ALERT - Motion Detected!" < /home/pi/FinalProject/FlaskWebServer/static/scripts/text.html'
subprocess.check_output(email, shell=True)
#print("Motion detected!")
else: # GPIO23 == low
msg = 'echo 0 > /home/pi/FinalProject/FlaskWebServer/static/scripts/pir.txt'
subprocess.check_output(msg, shell=True)
#print("End of motion detection event")
# PIR needs 30-60 seconds to initialize
for i in range(30):
time.sleep(1)
print(i)
try:
# When a change edge is detected on GPIO23, regardless of whatever else
# is happening in the program, the function MOTION will be run
GPIO.add_event_detect(PIR_PIN, GPIO.BOTH, callback=MOTION)
while 1:
time.sleep(100)
except KeyboardInterrupt:
print("Quit")
GPIO.cleanup()
pir.js
Javascript to dynamically update PIR status
//#################################################
//# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
//# ECE 5725: Embedded OS #
//# 04/23/2020 #
//# Lab 5: Final Project #
//#################################################
$(document).ready(function(){
var output = document.getElementById("pirStatus");
output.innerHTML = "";
//connect to the socket server.
var socket = io.connect('http://' + document.domain + ':' + location.port + '/pir');
//receive details from server
socket.on('pirStatus', function(msg) {
//console.log("PIR status = " + msg.pir);
//$('#pirStatus').html(msg.pir);
if(msg.pir == "Detected") {
var sentence = ''WARNING - The PIR sensor has detected motion!
"
var img = ""
output.innerHTML = sentence + img;
} else {
var sentence = "No motion has been detected.
"
var img = ""
output.innerHTML = sentence + img;
}
});
});
servoControl.py
Controls the pan and tilt servos
#!/usr/bin/env python
#################################################
# Avisha Kumar (ak754) & Tyler Sherman (tss86) #
# ECE 5725: Embedded OS #
# 04/23/2020 #
# Lab 5: Final Project #
#################################################
# start daemon = sudo pigpiod
# stop daemon = sudo killall pigpiod
import os
import errno
import time
import pigpio # uses Broadcom Numbering system
# Named Pipe
FIFO = '/home/pi/FinalProject/FlaskWebServer/static/scripts/servoFifo'
try:
os.mkfifo(FIFO)
except OSError as oe:
if oe.errno != errno.EEXIST:
raise
# Define Constants
servo_pan = 19 # GPIO19
servo_tilt = 18 # GPIO18
loc_tilt = 0 # location of servo_tilt in duty cycle
loc_pan = 0 # location of servo_pan in duty cycle
freq = 50 # Hz
right = 75000 # 7.5%
up = 75000 # 7.5%
center = 90000 # 9.0%
left = 105000 # 10.5%
down = 105000 # 10.5%
step = 500
#------------------SERVO FUNCTIONS------------------------#
def pan_left():
global loc_pan
if loc_pan <= left - step:
loc_pan += step
pi.hardware_PWM(servo_pan, freq, loc_pan)
print('servo_pan location = '+str(loc_pan))
else:
print('ERROR: servo_pan already left')
time.sleep(0.1)
def pan_right():
global loc_pan
if loc_pan >= right + step:
loc_pan -= step
pi.hardware_PWM(servo_pan, freq, loc_pan)
print('servo_pan location = '+str(loc_pan))
else:
print('ERROR: servo_pan already right')
time.sleep(0.1)
def tilt_up():
global loc_tilt
if loc_tilt >= up + step:
loc_tilt -= step
pi.hardware_PWM(servo_tilt, freq, loc_tilt)
print('servo_tilt location = '+str(loc_tilt))
else:
print('ERROR: servo_tilt already up')
time.sleep(0.1)
def tilt_down():
global loc_tilt
if loc_tilt <= down - step:
loc_tilt += step
pi.hardware_PWM(servo_tilt, freq, loc_tilt)
print('servo_tilt location = '+str(loc_tilt))
else:
print('ERROR: servo_tilt already down')
time.sleep(0.1)
def center_servos():
global loc_pan
global loc_tilt
pi.hardware_PWM(servo_pan, freq, center)
pi.hardware_PWM(servo_tilt, freq, center)
loc_pan = center
loc_tilt = center
print('centering servos')
#---------------------------------------------------------#
# Setup & initialize PWM
pi = pigpio.pi()
pi.hardware_PWM(servo_tilt, freq, center)
pi.hardware_PWM(servo_pan, freq, center)
loc_tilt = center
loc_pan = center
# Normally, opening the FIFO blocks until the other end is opened also
# This way, once the pipe is closed, the code will attempt to re-open it, which will block until another writer opens the pipe
running = True
while running:
#print("Opening FIFO...")
with open(FIFO) as fifo:
#print("FIFO opened")
while True:
cmd = (fifo.read())[0:1]
if cmd == 'u':
tilt_up()
elif cmd == 'l':
pan_left()
elif cmd == 'c':
center_servos()
elif cmd == 'r':
pan_right()
elif cmd == 'd':
tilt_down()
elif cmd == 'q':
running = False
break
# closes FIFO to limit CPU usafe
if len(cmd) == 0:
#print("Writer closed")
break
#print('Read: "{0}"'.format(data))
# Stop & cleanup
pi.stop()